Section: New Results

Shapes and contours

Riemannian metrics on spaces of curves and surfaces

Participant : Ian Jermyn [contact] .

This work is being done in collaboration with Anuj Srivastava of Florida State University [https://www.fsu.edu/ ].

Shape, Riemannian, metric, elastic, curve, surface, functional data, alignment

Statistical shape modelling has many applications in image processing and beyond. One of the key problems in this area is to develop and understand measures of shape similarity. One approach uses Riemannian metrics induced on `shape space' by Riemannian metrics on the space of embeddings. Current work is focused on generalizing to surfaces the elastic metric used for curve embeddings, and in finding surface representations that simplify computations in the same way that the square root velocity representation simplifies computations in the case of curves. The notion of a `square-root normal field' (SRNF), which leads to a reduced version of the full elastic metric, is a promising possibility in this direction.

The most recent work  [16] has focused on estimating the inverse of the SRNF map. If this can be done even approximately, a very efficient framework results: the surfaces, represented by their SRNFs, can be efficiently analyzed using standard Euclidean tools, and only the final results need to be mapped back to the surface space. In this work, we developed a procedure for inverting SRNF maps of star-shaped surfaces, a special case for which analytic results can be obtained. We tested our method via the classification of 34 cases of ADHD (Attention Deficit Hyperactivity Disorder), plus controls, in the Detroit Fetal Alcohol and Drug Exposure Cohort study. We obtained state-of-the-art results.

Enforcing monotonous shape growth or shrinkage in video segmentation

Participant : Yuliya Tarabalka [contact] .

This work has been done in collaboration with Dr. Guillaume Charpiat (STARS team, Inria-SAM), Dr. Bjoern Menze (Technische Universität München, Germany and Asclepios team, Inria-SAM), and Dr. Ludovic Brucker (NASA GSFC, USA) [http://www.nasa.gov ].

Video segmentation, graph cut, shape analysis, shape growth

Automatic segmentation of objects from video data is a difficult task, especially when image sequences are subject to low signal-to-noise ratio or low contrast between the intensities of neighboring structures. Such challenging data are acquired routinely, for example, in medical imaging or satellite remote sensing. While individual frames can be analyzed independently, temporal coherence in image sequences provides a lot of information not available for a single image. In this work, we focused on segmenting shapes that grow or shrink monotonically in time, from sequences of extremely noisy images.

We proposed a new method for the joint segmentation of monotonically growing or shrinking shapes in a time sequence of images with low signal-to-noise ratio [3] . The task of segmenting the image time series is expressed as an optimization problem using the spatio-temporal graph of pixels, in which we are able to impose the constraint of shape growth or shrinkage by introducing unidirectional infinite-weight links connecting pixels at the same spatial locations in successive image frames. The globally-optimal solution is computed with a graph-cut algorithm. The performance of the proposed method was validated on three applications: segmentation of melting sea ice floes; of growing burned areas from time series of 2D satellite images; and of a growing brain tumor from sequences of 3D medical scans. In the latter application, we imposed an additional inter-sequences inclusion constraint by adding directed infinite-weight links between pixels of dependent image structures. Figure 8 shows a multi-year sea ice floe segmentation result. The proposed method proved to be robust to high noise and low contrast, and to cope well with missing data. Moreover, it showed linear complexity in practice.

Multi-label image segmentation with partition trees and shape prior

Participants : Emmanuel Maggiori, Yuliya Tarabalka [contact] .

This work has been done in collaboration with Dr. Guillaume Charpiat (STARS team, Inria-SAM).

Partition trees, multi-class segmentation, shape priors, graph cut

The multi-label segmentation of images is one of the great challenges in computer vision. It consists in the simultaneous partitioning of an image into regions and the assignment of labels to each of the segments. The problem can be posed as the minimization of an energy with respect to a set of variables which can take one of multiple labels. Throughout the years, several efforts have been done in the design of algorithms that minimize such energies.

We proposed a new framework for multi-label image segmentation with shape priors using a binary partition tree [19] . In the literature, such trees are used to represent hierarchical partitions of images, and are usually computed in a bottom-up manner based on color similarities, then processed to detect objects with a known shape prior. However, not considering shape priors during the construction phase induces mistakes in the later segmentation. This paper proposes a method which uses both color distribution and shape priors to optimize the trees for image segmentation. The method consists in pruning and regrafting tree branches in order to minimize the energy of the best segmentation that can be extracted from the tree. Theoretical guarantees help reducing the search space and make the optimization efficient (see Figure 9 (i)) and [19] . Our experiments (see Figure 9 ) show that the optimization approach succeeds in incorporating shape information into multi-label segmentation, outperforming the state-of-the-art.